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Preliminaries 


We begin by recalling some basic notions of functional analysis. A 
measurable function f belongs to the Lebesgue space L, (IR), 1 < p< 
if 

Equation: 


Willy = (fF @)Pae)" <o. 


e.@) 


A Hilbert space is a space where an inner product is defined. In particular, 
the space Dz (IR) is a Hilbert space, where the inner product of 2 functions 
f and g is defined as: 

Equation: 


+00 
<ji.¢go= f (x) g(a)da. 


—oo 


In this presentation we work with functions defined on RR, but that take 
values in QO. 1lex]0.05em1.25ex . Hence g(x) denotes the complex 
conjugate of g(a). We say that 2 functions are orthogonal if their inner 
product is zero. A function is Hélder continuous of order a, (0 < a < 1) at 
point z if : 

Equation: 


If (x) — f(e@+h)| =O(A*). 


Multiresolution analysis 


The scaling function and the subspaces 


There are two ways to introduce wavelets: one is through the continuous 
wavelet transform, and the other is through multiresolution analysis (MRA), 
which is the presentation adopted here. Here we start by defining 
multiresolution analysis and thereafter we give one example of such MRA. 


Definition 


Note: ( Multiresolution analysis) A multiresolution analysis of D2 (IR) is 
defined as a sequence of closed subspaces V; C Lz (IR), 7 € Z with the 
following properties: 


—_— 


. Equation: 
aie Vise Voc. Vics 


2. The spaces V; satisfy 
Equation: 


U V; is dense in D2 (IR) and () V,={0} 


jE jE 


3.1f f(z) EV, f (2/2) € V;. This property means that all the spaces 
V; are scaled versions of the central space Vo. 

A.If f © Vo, f (.—k) © Vo, k © ZZ. That is, Vo(and hence all the V;) is 
invariant under translation. 

5. There exists @ € Vo such that {¢9,,;7 € ZZ} is an orthonormal basis 
in Vo. 


Condition 5 in [link] seems to be quite contrived, but it can be relaxed 
(i.e.,instead of taking orthonormal basis, we can take Riesz basis). We will 
use the following terminology: a level of a multiresolution analysis is one 
of the V; subspaces and one level is coarser (respectively finer) with 
respect to another whenever the index of the corresponding subspace is 
smaller (respectively bigger). 


Consequence of the definition 


Let us make a couple of simple observations concerning this definition. 
Combining the facts that 


1. Equation: 
d(x) € Vo 


2. {(. —k), k € Z} is an orthonormal basis for Vo 
3. @ (2/2) E V,, 


we obtain that, for fixed j, { bj (e)= 23/24 (Ma — k),k & ZL‘ is an 
orthonormal basis for V3. 


Since ¢ € Vo C Vj, we can express ¢ as a linear combination of {¢1,x} : 
Equation: 


(x) 


So hdr (2) 
k 
V2S— hyd (2a — k). 
k 


[link] is called the refinement equation, or the two scales difference 
equation. The function ¢(2) is called the scaling function. Under very 
general condition, ¢ is uniquely defined by its refinement equation and the 
normalisation 

Equation: 


+00 
@(a)dz = 1. 


—oo 


The spaces V; will be used to approximate general functions (see an 
example below). This will be done by defining appropriate projections onto 
these spaces. Since the union of all the V; is dense in Ly (IR), we are 
guaranteed that any given function of D2 can be approximated arbitrarily 
close by such projections, i.e.: 

Equation: 


lim #Y;f =f, 
jroo 


for all f in Ly. Note that the orthogonal projection of f onto V; can be 
written as: 
Equation: 


Pf = J- andjr- 


kEZZ, 


where a, =< f, Oj 4 >- 


Example 


The simplest example of a scaling function is given by the Haar function: 
Equation: 

1 if0<a2<1 

0 otherwise 


¢ (x) = Dp = { 


Hence we have that 
Equation: 


1 if0<a<1/2 
0 otherwise 


(22) =4 


and 

Equation: 

1 if1/2<a<1 
0 otherwise 


$e -1)={ 


The function ¢ generates, by translation and scaling, a multiresolution 
analysis for the spaces V; defined by: 
Equation: 


Vj; ={f € D2 IR); Vk © Z, f | 2%%,27(n+1, = constant } 


The wavelet function and the detail spaces Wj 


The detail space Wj 


Rather than considering all our nested spaces V;, we would like to code 
only the information needed to go from V; to V;,1. Hence we define by W; 
the space complementing V; in V;+1: 

Equation: 


Vin = Vj OW, 


This space W;; answers our question: it contains the “detail” information 
needed to go from an approximation at resolution 7 to an approximation at 
resolution 7 + 1. Consequently, by using recursively the [link], we have: 
Equation: 


BjeunW; = Lo GR). 


The main interest of MRA lies in the fact that, whenever a collection of 
closed subspaces satisfies the conditions in [link], there exists an 
orthonormal wavelet basis, noted {7;;,,k € ZZ} of W;. Hence, we can say 
in very general terms that the projection of a function f onto V;,1 can be 
decomposed as 

Equation: 


Piif = Aft Qyf, 


where #, is the projection operator onto V;, and Q; is the projection 
operator onto W;. Before proceeding further, let us state clearly what the 
term “orthogonal wavelet ” involves. 


Recapitulation: orthogonal wavelet 


We introduce above the class of orthogonal wavelets. Let us define this 
precisely. 


An orthogonal wavelet basis is associated with an orthogonal 
multiresolution analysis that can be defined as follows. We talk about 
orthogonal MRA when the wavelet spaces W;; are defined as the 
orthogonal complement of V; in V;,1. Consequently, the spaces W,, with 
j € Z are all mutually orthogonal, the projections #; and 2; are 
orthogonal, so that the expansion 

Equation: 


f(z) =) / Sif (2) 


is orthogonal. Moreover, as 2; is orthogonal, the projection onto W; can 
explicitely be written as: 
Equation: 


Qin) = S— Bir Dir (x), 
k 


with 
Equation: 


Bir =< fF, Vik >. 


A sufficient condition for a MRA to be orthogonal is: 
Equation: 


WoLVo, 


or < ~, ¢(. —l) >= 0 for! € Z, since the other conditions simply follow 
from scaling. 


An orthogonal wavelet is a function w such that the collection of functions 
{w(a — 1)|l € ZZ} is an orthonormal basis of Wo. This is the case if 


<W,¥(.-l) >= dio. 


In the following, we consider only orthogonal wavelets. We now outline 
how to construct a wavelet function (a) starting from $(«), and thereafter 
we show what this construction gives with the Haar function. 


How to construct W(x) starting from p(x)? 


Suppose we have an orthonormal basis (ONB) {¢;,4, & € ZZ} for V; and we 
want to construct wj, such that 


¢ W5r,k € ZZ form an ONB for W; 
e Equation: 


V;LW;j, ie. < O54, Pin >= 0 Vk, k’ 


e Equation: 


W;LW; for 7 #7’. 


It is natural to use conditions given by the MRA aspect to obtain this. More 
specifically, the following relationships are used to characterize w : 


1. Since @ € Vo C Vj, and the $1, are an ONB in Vj, we have: 
Equation: 


d(x) = So habia, he =< 0, Pk >> |ny|? = 1. 
k 


keEZZ, 


(refinement equation) 
2. Equation: 


So = | 8(0)6(e ~ bda 


(orthonormality of ¢(. —k)) 

3. Let us now characterize Wg : f € Wo is equivalent to f € V; and 
f LVo. Since f € Vi, we have: 
Equation: 


ci — b> frPin, withf,, =< Ne Pin >. 


The constraint f_1Vo is implied by flo, for all k. 

4. Taking the general form of f € Wo, we can deduce a candidate for our 
wavelet. We then need to verify that the wo, are indeed an ONB of 
Wo. 


In fact, in our setting, the conditions given above can be re-written in the 
Fourier domain, where the manipulations become easier (for details, see 
[link], chapter 5). Let us now state the result of these manipulations. 


Note:(Daubechies, chap 5) If a ladder of closed subspaces (Vj) 5m in 


L (IR) satisfies the conditions of the [link], then there exists an associated 
orthonormal wavelet basis {W;,4; 7, & € ZZ} for Lz (IR) such that: 


Equation: 


Pig = PAD) <Die > Pie 
k 


One possibility for the construction of the wavelet ~ is to take 
Equation: 


(a) = (2) 33 (C1)"hi-nd (22 — n) 


n 


(with convergence of this serie is [2 sense). 


Example(continued) 


Let us see what the recipe of [link] gives for the Haar multiresolution 
analysis. In that case, (2) = 1 for0 < x < 1,0 otherwise. Hence: 
Equation: 


1/V2 ifn=0,1 
0 otherwise 


= V2 [ ded (2)6 (20 —n)= { 


Consequently, 
Equation: 


W(t) = V2hi¢ (2x) — V2hod (2x — 1) 
= $(22) — d(2e -1) 

1 if0<2<1/2 

—1 if1/2<a<1 

0 otherwise 


Homogeneous and inhomogeneous representation of 


Inhomogeneous representation 


If we consider a first (coarse) approximation of f € Vo, and then “refine” 
this approximation with detail spaces W,;, the decomposition of f can be 
written as: 

Equation: 


f = Aft SY Bif 
4=0 


= dcxbox (2 y+ SI SS Bi adar ( 
k 


j=0 


where ag =< f, 0,4 > and Bj, =< f, jx >. In this case, we talk about 
inhomogeneous representation of f. 


Homogeneous representation 


If we use the fact that 
Equation: 


®,;<zW; is dense in Lp (IR), 


we can decompose f as a linear combination of functions ~,;, only: 
Equation: 


+00 


f(t) = D7 >) Biabix (2). 
k 


j=—00 


We then talk about homogeneous representation of f. 


Properties of the homogeneous representation 


e Each coefficient 6; in [Link] depends only locally on f because 
Equation: 


bee / f (Wer (a)de, 


and the wavelet 7, ;, (z) has (essentially) bounded support. 

¢ 8; gives information on scale 2-/, near position 2 ’k 

e a discontinuity in f only affects a small proportion of coefficients— a 
fixed number at each frequency level. 


The filtered transform 


Introduction 


We saw in a previous section an example of a scaling function @, and we 
outlined how to construct ~(also called the mother wavelet) starting from @ 
(the father wavelet). Suppose now we have at our disposal {¢,;,} and 
{w; x}. In fact, it is sufficient for our purpose to know the value of these 
functions at dyadic points 2-’k, 7 € ZZ, k € ZZ. We would like to compute 
in an efficient way the wavelet representation described in Homogeneous 
and inhomogeneous representation of f, that is, we would like to have a fast 
algorithm to compute the wavelet coefficients. 


Filter algorithm-Fast wavelet transform 


We will still use the relationship between the functions spaces V; and W; to 
find a fast wavelet transform (FWT). We start by recalling that, since both 
the scaling function @ € Vo and the wavelet w € Wo are in V;, and since 


Vj is generated by $1, = /2¢ (22 — k),k © ZZ, there exist two sequences 
{hz} and {g,} € I? such that 
Equation: 


(x) = V2S~ hyd (22 — k) 
k 


Equation: 


b (x) = V2 0%; 9nd (2a — k) 


for all x € IR. On the other hand, we know that Vi = Vo @ Wo and as we 
consider the orthogonal case, it follows immediately that : 
Equation: 


V2 (2a) = > [hand (@ — k) + gone (2 — f)] 


k 


Equation: 


V 29 (22 — 1) = >, [Ri_onb (@ — k) + gon (a — Bd). 


These two equations [link] and [link] can be combined into a single 
formula: 
Equation: 


V2¢ (2x — 1) = S¢ [hi_-anb (@ — k) + giant (a — k)], Le Z, 


k 


which is called the “decomposition relation” of @ and w. 


Note that, in the bi-orthogonal case there are four sequences in /? instead of 
two (denoted here by {hz} and {g;,}): we have two sequences for the 2- 
scales relations [link], [link], and two others for the decomposition relations 
[link], [link]. In the following algorithm, we drop the normalisation 
constant. Suppose that we want to decompose f as a sum of wavelets and 
that we have computed, or are given, the inner products of f with @;,, 
where J is the finest scale we can work on. We denote these inner products 
by c’. Now, our task is to compute c}, and d/.,, 7 < J, where 

Equation: 


Ff = S> ch (2’x — k);c} =< f’, dn > 
k 
Equation: 


Dif =Y, dip (2a — kb); dh =< fl, bie > 


Decomposition algorithm 


By combining [link], [link] and [link], we get (see [link]): 


Equation: 


‘an ee N } j 
ie hi 2RKcy 
l 


os) oe J 
dj, => Jl—2kC} « 


bb) 


Observe that both c3? and d3~? are obtained from c} by “moving average 
schemes, using the decomposition sequence as “weights”, with the 
exception that these moving averages are sampled only at the even integers. 
This is called downsampling. 


Reconstruction algorithm 


In the orthogonal case, the reconstruction algorithm follows easily from the 
relationships: 
Equation: 
+1 +1 
Cy =< LP P4142 > 


Gls Pea f= Pi f+ 2sf 


= se cLb;h a De di abs ke, 


which gives: 
Equation: 


eh 
c= oe < Pj,kr Pjtin > + 5 i < Wits Pi41)0 > 
k k 


k 


Hence cJ+! is obtained from ce) and dj by two moving average. 


Mallat's algorithm 


In the previous section, we assumed that we knew the coefficients 
Equation: 


ch =< fbr > ke Z. 


The question to ask is: how to compute these coefficients ? In Mallat's 
algorithm (see [link]), we consider that the finest scale is constituted by the 
observations{Y;,}7_, themselves. To use the MRA presented above, these 
observations must be taken at equispaced points, i.e. we can write that 


Equation: 
k n 
Yo 4 = — : 
Fim Ca), 


Moreover, we assume that n (the number of observations) is a power of 2 : 
n = 2/7, with J denoting the finest level. 


Mallat's algorithm is based on the fact that, as 7 tends to infinity, the support 
of @;, tends to become smaller and smaller. We have: 


Equation: 
lim ¢;% (2) > 6 fi = Lae kh) 
Him. 3,4 (@ | Da he 
= 0 otherwise. 
Hence Mallat considered that we can compute cj, as: 
Equation: 


od ~ [ F@se ere 


The starting point of this algorithm is thus extremely simple: we just take as 
value for ce the whole set of observations. Thereafter, having constructed 
the filters {h;,} and {g;} (or, equivalently, having constructed ¢ and 7), 
we can compute a fast wavelet transform using the algorithm presented in 
the previous section. 


Approximation of functions with wavelets 


Approximation of functions with wavelets 


In Example from Multiresolution analysis, we saw the Haar wavelet basis. 
This is the simplest wavelet one can imagine, but its approximation 
properties are not very good. Indeed, the accuracy of the approximation is 
somehow related to the regularity of the functions w= and ¢. We will show 
this for different settings: in case the function f is continuous, belongs to a 
Sobolev or to a Hélder space. But first we introduce the notion of regularity 
of a MRA. 


Regularity of a multiresolution analysis 
For wavelet bases (orthonormal or not), there is a link between the 


regularity of ~ and the number of vanishing moments. More precisely, we 
have the following: 


Note: Let {7,;,} be an orthonormal basis (ONB) in L? (IR), with 
w € @.1ex]0.05em1. 25ex ™, bounded for 1 < m and 
|b (x)| < C(1+ |x|) ° for a > m +1. Then we have: 


Equation: 


[oe (x)dx = 0 for 1 =0,1,...,m. 


(this describes the “decay in frequency domain”). 


This proposition implies the following corollary: 


Note: Suppose the {~,;,} are orthonormal. Then it is impossible that w has 
exponential decay, and that ~ € @. lex|0.05em1. 25ex ©, with all the 
derivatives bounded, unless = = 0 


This corollary tells us that a trade-off has to be done: we have to choose for 
exponential (or faster) decay in, either time or frequency domain; we 
cannot have both. We now come to the definition of a r—regular MRA (see 
[link]). 


Note: (Meyer, 1990) A MRA is called r-regular (r € IN) when 
@ € WM. 1lex]0.05em1. 25ex " and for all m € WN, there exists a constant 
Cy, > 0 such that: 


Equation: 


0° (x) 


ce I< Cm(1 + |x|) “Va <r 


If one has a r—regular MRA, then the corresponding wavelet 

w(x) € M.1ex|0.05em1. 25ex ", satisfies [link] and has r vanishing 
moments: 

Equation: 


[ot (ide =0,k= Ua th 


We now have the tools needed to measure the decay of approximation error 
when the resolution (or the finest level) increases. 


Approximation of a continuous function 


Note: Let {~;,} come from a r-regular MRA. Then, we have, for a 
continuous function f € (0. 1lex]0.05em1. 25ex *(0 < s < r)the 
following: 


Equation: 


|< f, Wik = | — QO (seer) 


Note: (s = 1). As f (x) = 0 and |f (x) — f (k/27)|< Cla — k/2)|, (f 


is Lipschitz continuous), we have: 


Equation: 


IK fidae>l = 2? f [F@)-F(e/2!)|w (242 - Kdz|,ke Z 


IA 


cir f x — k/2) || (2x — k)|dz 

z cain f aie — kllab (24x — k) |e 
( let u = 2x —k) 

= zag f u w(u)|du < co 


_ gare): 


For s > 1, we use proposition [link] and we iterate. L] 


Note that the reverse of [link] is true: [link] entails that f is in C’*. 


Note: Under the assumptions of [link], the error of approximation of a 
function f € QO. 1lex]0.05em1. 25ex * at scale V; is given by: 


Equation: 


Wf- >> <fibsp > oralle =O (27°) 
k 


Note: Using the exact decomposition of f given by Equation 30 from 
Multiresolution Analysis, one has: 


Equation: 
f- Psflly = ISOS 5 <f,vin > Viale 
fod k 
< Scr et) | So dealle 
jd k 


Now, if we suppose that | (k)| < C’(1 + |k|) ,(m > s+4+1), we 
obtain: 
Equation: 


| > Vaal, < C12"? 
k 


Putting inequalities [link] and [link] together, we have: 
Equation: 


If - Pafll, $C" 12 * =0"(1—- 2") V2" =O") 


jed 


Hence, we verify with this last corollary that, as J increases, the 
approximation of f becomes more accurate. 


Approximation of functions in Sobolev spaces 


Let us first recall the definition of weak differentiability, for this notion 
intervenes in the definition of a Sobolev space. 


Note: Let f be a function defined on the real line which is integrable on 
every bounded interval. If there exists a function g defined on the real line 
which is integrable on every bounded interval such that: 


Equation: 


Va <y, [ g(widu= fly ~F (2), 


then the function f is called weakly differentiable. The function g is defined 
almost everywhere, is called the weak derivative of f and will be denoted 


by f’. 


Note: A function f is V-times weakly differentiable if it has derivatives 
f, fF’, 0; f~) which are continuous and f “) which is a weak 
derivative. 


We are now able to define Sobolev spaces. 


Note: Let 1 < p < co,m € {0,1,...}. The function f € L, (IR) belongs 
to the Sobolev space W,” (IR) if it is m-times weakly differentiable, and if 


f'™ € L, (IR). In particular, Ww, (IR) = L, (IR). 


The approximation properties of wavelet expansions on Sobolev spaces are 
given, among other, in Hardle et.al (see [link]). Suppose we have at our 
disposal a scaling function @ which generates a MRA. The approximation 
theorem can be stated as follows: 


Note: (Approx. in Sobolev space) Let ¢ be a scaling function such that 
{o(. —k), k € Z} is an ONB and the corresponding spaces V; are nested. 
In addition, let ¢@ be such that 


Equation: 


N+1 
/ d(x) x dz < o, 


and let at least one of the following assumptions hold: 


1. Equation: 


¢ € W.. (R) for some 1 < q < co 


2. fx" (x)dx = 0,n = 0,1,..., N, where ¢ is the mother wavelet 
associated to @. 


Then, if f belongs to the Sobolev space Ww,” *1 (IR), we have: 
Equation: 


Piaf - fll, =O (2%), as j + 00, 


where #, is the projection operator onto V;. 


Approximation of functions in Holder spaces 


Here we assume for simplicity that ~% has compact support and is 
QO. 1ex]0.05em1. 25ex | (the formulation of the theorems are slightly 
different for more general 7). 


Note: If f is Hélder continuous with exponent a,0 < a < 1at Zo, then 


Equation: 


max {|< f, Vin > dist(xo, supp (bj,n)) °} =O (2-4/0) 


The reverse of theorem [link] does not exactly hold: we must modify 
condition [link] slightly. More precisely, we have the following: 


Note: Define, for « > 0, the set 


Equation: 


S (x0, J; €) a {k € ZZ| supp (23,4) Lo — €,%o Te [A O}. 


If, for some € > 0 and some a(0 < a < 1), 
Equation: 


max I< bie Wik >| = O 2te)), 
kES(z0,J,€) 


then f is Hélder continuous with exponent a@ at xo. 


